If the image differs, but the number of pixels that differ is no more
than tolerated-diff-pixels and the differences are no more than
tolerated-diff-level, then we treat it as a success with warnings, save
the .diff.png for analysis, and use g_test_incomplete() to record the
test-case as "TODO".
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftests-Allow-minor-differences-to-be-tolerated.patch
Simon McVittie [Sat, 13 Feb 2021 18:26:24 +0000 (18:26 +0000)]
reftest_compare_surfaces: Report how much the images differ
In unattended/non-interactive/autobuilder environments where the images
are not trivially accessible, this provides a way to distinguish between
totally different rendering and more subtle issues.
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftest_compare_surfaces-Report-how-much-the-images-diffe.patch
Simon McVittie [Tue, 6 Dec 2022 11:13:59 +0000 (11:13 +0000)]
node-editor: Save test data relative to current working directory
This avoids hard-coding the GTK source directory into the binary, which
would make the binary non-reproducible across different source paths,
particularly in a distro build environment where the source directory
used by autobuilders will often not exist on end-user systems. The node
editor can still be used to create new test-cases by running it with the
GTK source directory as its current working directory.
Bug: https://gitlab.gnome.org/GNOME/gtk/-/issues/5403
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/5297 Signed-off-by: Simon McVittie <smcv@debian.org>
Gbp-Pq: Name node-editor-Save-test-data-relative-to-current-working-di.patch
Simon McVittie [Wed, 23 Nov 2022 21:26:50 +0000 (21:26 +0000)]
testsuite: Don't create .test files for flaky or failing tests
These tests can be run manually, but are not suitable for use as an
acceptance test, so let's not make frameworks like Debian's autopkgtest
run these when they run ginsttest-runner in the most obvious way.
Simon McVittie [Wed, 23 Nov 2022 19:13:32 +0000 (19:13 +0000)]
testsuite: Use separate setups for unstable tests instead of should_fail
There are two possible interpretations of "expected failure": either
the test *must* fail (exactly the inverse of an ordinary test, with
success becoming failure and failure becoming success), or the test
*may* fail (with success intended, but failure possible in some
environments). Autotools had the second interpretation, which seems
more useful in practice, but Meson has the first.
Instead of using should_fail, we can put the tests in one of two new
suites: "flaky" is intended for tests that succeed or fail unpredictably
according to the test environment or chance, while "failing" is for
tests that ought to succeed but currently never do as a result of a
bug or missing functionality. With a sufficiently new version of Meson,
the flaky and failing tests are not run by default, but can be requested
by running a setup that does not exclude them, with a command like:
meson test --setup=x11_unstable --suite=flaky --suite=failing
As a bonus, now that we're setting up setups and their excluded suites
programmatically, the gsk-compare-broadway tests are also excluded by
default when running the test setup for a non-broadway backend.
When running the tests in CI, --suite=gtk overrides the default
exclude_suites, so we have to specify --no-suite=flaky and
--no-suite=failing explicitly.
This arrangement is inspired by GNOME/glib!2987, which was contributed
by Marco Trevisan.
Carlos Garnacho [Sat, 4 Feb 2023 12:10:42 +0000 (13:10 +0100)]
gdk/x11: Reset all scroll valuators on enter
We no longer need to make much distinction between multiple logical
devices, plus it breaks esp. with the Xwayland input device distribution.
Just iterate across all devices and reset their scroll valuators.
The owner_events=TRUE grab makes GDK on X11 see events happening
outside every client window as received on the grab window.
Additionally check that the pointer is inside the grab window
(i.e. it received GDK_CROSSING_NORMAL crossing events for the
core pointer) in order to handle clicks happening outside client
windows.
These new paths are expected to be a no-op on Wayland, and to
also work for touchscreen input on X11, due to emulated pointer
events.
If the image differs, but the number of pixels that differ is no more
than tolerated-diff-pixels and the differences are no more than
tolerated-diff-level, then we treat it as a success with warnings, save
the .diff.png for analysis, and use g_test_incomplete() to record the
test-case as "TODO".
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftests-Allow-minor-differences-to-be-tolerated.patch
Simon McVittie [Sat, 13 Feb 2021 18:26:24 +0000 (18:26 +0000)]
reftest_compare_surfaces: Report how much the images differ
In unattended/non-interactive/autobuilder environments where the images
are not trivially accessible, this provides a way to distinguish between
totally different rendering and more subtle issues.
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftest_compare_surfaces-Report-how-much-the-images-diffe.patch
Carlos Garnacho [Sat, 4 Feb 2023 12:10:42 +0000 (13:10 +0100)]
gdk/x11: Reset all scroll valuators on enter
We no longer need to make much distinction between multiple logical
devices, plus it breaks esp. with the Xwayland input device distribution.
Just iterate across all devices and reset their scroll valuators.
Simon McVittie [Tue, 6 Dec 2022 11:13:59 +0000 (11:13 +0000)]
node-editor: Save test data relative to current working directory
This avoids hard-coding the GTK source directory into the binary, which
would make the binary non-reproducible across different source paths,
particularly in a distro build environment where the source directory
used by autobuilders will often not exist on end-user systems. The node
editor can still be used to create new test-cases by running it with the
GTK source directory as its current working directory.
Bug: https://gitlab.gnome.org/GNOME/gtk/-/issues/5403
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/5297 Signed-off-by: Simon McVittie <smcv@debian.org>
Gbp-Pq: Name node-editor-Save-test-data-relative-to-current-working-di.patch
Simon McVittie [Wed, 23 Nov 2022 21:26:50 +0000 (21:26 +0000)]
testsuite: Don't create .test files for flaky or failing tests
These tests can be run manually, but are not suitable for use as an
acceptance test, so let's not make frameworks like Debian's autopkgtest
run these when they run ginsttest-runner in the most obvious way.
Simon McVittie [Wed, 23 Nov 2022 19:13:32 +0000 (19:13 +0000)]
testsuite: Use separate setups for unstable tests instead of should_fail
There are two possible interpretations of "expected failure": either
the test *must* fail (exactly the inverse of an ordinary test, with
success becoming failure and failure becoming success), or the test
*may* fail (with success intended, but failure possible in some
environments). Autotools had the second interpretation, which seems
more useful in practice, but Meson has the first.
Instead of using should_fail, we can put the tests in one of two new
suites: "flaky" is intended for tests that succeed or fail unpredictably
according to the test environment or chance, while "failing" is for
tests that ought to succeed but currently never do as a result of a
bug or missing functionality. With a sufficiently new version of Meson,
the flaky and failing tests are not run by default, but can be requested
by running a setup that does not exclude them, with a command like:
meson test --setup=x11_unstable --suite=flaky --suite=failing
As a bonus, now that we're setting up setups and their excluded suites
programmatically, the gsk-compare-broadway tests are also excluded by
default when running the test setup for a non-broadway backend.
When running the tests in CI, --suite=gtk overrides the default
exclude_suites, so we have to specify --no-suite=flaky and
--no-suite=failing explicitly.
This arrangement is inspired by GNOME/glib!2987, which was contributed
by Marco Trevisan.
The owner_events=TRUE grab makes GDK on X11 see events happening
outside every client window as received on the grab window.
Additionally check that the pointer is inside the grab window
(i.e. it received GDK_CROSSING_NORMAL crossing events for the
core pointer) in order to handle clicks happening outside client
windows.
These new paths are expected to be a no-op on Wayland, and to
also work for touchscreen input on X11, due to emulated pointer
events.
- Optimize column views by not creating list item widgets in columns
that are not visible (GNOME/gtk!5374)
- In X11, make sure menus, popovers etc. don't interfere with ability
to click/select in other applications' windows
(GNOME/gtk#5364, GNOME/gtk#5400)
- Do not mark non-user-facing strings as translatable (GNOME/gtk#5146)
- Fix animations and transitions randomly stopping (GNOME/gtk#4426)
- Translation updates: ab, tr
If the image differs, but the number of pixels that differ is no more
than tolerated-diff-pixels and the differences are no more than
tolerated-diff-level, then we treat it as a success with warnings, save
the .diff.png for analysis, and use g_test_incomplete() to record the
test-case as "TODO".
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftests-Allow-minor-differences-to-be-tolerated.patch
Simon McVittie [Sat, 13 Feb 2021 18:26:24 +0000 (18:26 +0000)]
reftest_compare_surfaces: Report how much the images differ
In unattended/non-interactive/autobuilder environments where the images
are not trivially accessible, this provides a way to distinguish between
totally different rendering and more subtle issues.
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftest_compare_surfaces-Report-how-much-the-images-diffe.patch
Simon McVittie [Tue, 6 Dec 2022 11:13:59 +0000 (11:13 +0000)]
node-editor: Save test data relative to current working directory
This avoids hard-coding the GTK source directory into the binary, which
would make the binary non-reproducible across different source paths,
particularly in a distro build environment where the source directory
used by autobuilders will often not exist on end-user systems. The node
editor can still be used to create new test-cases by running it with the
GTK source directory as its current working directory.
Bug: https://gitlab.gnome.org/GNOME/gtk/-/issues/5403
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/5297 Signed-off-by: Simon McVittie <smcv@debian.org>
Gbp-Pq: Name node-editor-Save-test-data-relative-to-current-working-di.patch
Simon McVittie [Wed, 23 Nov 2022 21:26:50 +0000 (21:26 +0000)]
testsuite: Don't create .test files for flaky or failing tests
These tests can be run manually, but are not suitable for use as an
acceptance test, so let's not make frameworks like Debian's autopkgtest
run these when they run ginsttest-runner in the most obvious way.
Simon McVittie [Wed, 23 Nov 2022 19:13:32 +0000 (19:13 +0000)]
testsuite: Use separate setups for unstable tests instead of should_fail
There are two possible interpretations of "expected failure": either
the test *must* fail (exactly the inverse of an ordinary test, with
success becoming failure and failure becoming success), or the test
*may* fail (with success intended, but failure possible in some
environments). Autotools had the second interpretation, which seems
more useful in practice, but Meson has the first.
Instead of using should_fail, we can put the tests in one of two new
suites: "flaky" is intended for tests that succeed or fail unpredictably
according to the test environment or chance, while "failing" is for
tests that ought to succeed but currently never do as a result of a
bug or missing functionality. With a sufficiently new version of Meson,
the flaky and failing tests are not run by default, but can be requested
by running a setup that does not exclude them, with a command like:
meson test --setup=x11_unstable --suite=flaky --suite=failing
As a bonus, now that we're setting up setups and their excluded suites
programmatically, the gsk-compare-broadway tests are also excluded by
default when running the test setup for a non-broadway backend.
When running the tests in CI, --suite=gtk overrides the default
exclude_suites, so we have to specify --no-suite=flaky and
--no-suite=failing explicitly.
This arrangement is inspired by GNOME/glib!2987, which was contributed
by Marco Trevisan.
[changes brought forward from 4.4.0+ds1-5+rpi1 by Peter Michael Green <plugwash@raspbian.org> at Tue, 07 Sep 2021 10:26:09 +0000]
* Disable a couple of faliing tests in Raspbian.
* Fix clean target.
[changes introduced in 4.8.3+ds-1+rpi1 by Peter Michael Green]
* Disable a couple more failing tests.
If the image differs, but the number of pixels that differ is no more
than tolerated-diff-pixels and the differences are no more than
tolerated-diff-level, then we treat it as a success with warnings, save
the .diff.png for analysis, and use g_test_incomplete() to record the
test-case as "TODO".
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftests-Allow-minor-differences-to-be-tolerated.patch
Simon McVittie [Sat, 13 Feb 2021 18:26:24 +0000 (18:26 +0000)]
reftest_compare_surfaces: Report how much the images differ
In unattended/non-interactive/autobuilder environments where the images
are not trivially accessible, this provides a way to distinguish between
totally different rendering and more subtle issues.
Signed-off-by: Simon McVittie <smcv@debian.org>
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/3195
Applied-upstream: no, upstream want reftests to be a strict pass/fail with identical results required
Gbp-Pq: Topic debian
Gbp-Pq: Name reftest_compare_surfaces-Report-how-much-the-images-diffe.patch
Simon McVittie [Tue, 6 Dec 2022 11:13:59 +0000 (11:13 +0000)]
node-editor: Save test data relative to current working directory
This avoids hard-coding the GTK source directory into the binary, which
would make the binary non-reproducible across different source paths,
particularly in a distro build environment where the source directory
used by autobuilders will often not exist on end-user systems. The node
editor can still be used to create new test-cases by running it with the
GTK source directory as its current working directory.
Bug: https://gitlab.gnome.org/GNOME/gtk/-/issues/5403
Forwarded: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/5297 Signed-off-by: Simon McVittie <smcv@debian.org>
Gbp-Pq: Name node-editor-Save-test-data-relative-to-current-working-di.patch
Simon McVittie [Wed, 23 Nov 2022 21:26:50 +0000 (21:26 +0000)]
testsuite: Don't create .test files for flaky or failing tests
These tests can be run manually, but are not suitable for use as an
acceptance test, so let's not make frameworks like Debian's autopkgtest
run these when they run ginsttest-runner in the most obvious way.
Simon McVittie [Wed, 23 Nov 2022 19:13:32 +0000 (19:13 +0000)]
testsuite: Use separate setups for unstable tests instead of should_fail
There are two possible interpretations of "expected failure": either
the test *must* fail (exactly the inverse of an ordinary test, with
success becoming failure and failure becoming success), or the test
*may* fail (with success intended, but failure possible in some
environments). Autotools had the second interpretation, which seems
more useful in practice, but Meson has the first.
Instead of using should_fail, we can put the tests in one of two new
suites: "flaky" is intended for tests that succeed or fail unpredictably
according to the test environment or chance, while "failing" is for
tests that ought to succeed but currently never do as a result of a
bug or missing functionality. With a sufficiently new version of Meson,
the flaky and failing tests are not run by default, but can be requested
by running a setup that does not exclude them, with a command like:
meson test --setup=x11_unstable --suite=flaky --suite=failing
As a bonus, now that we're setting up setups and their excluded suites
programmatically, the gsk-compare-broadway tests are also excluded by
default when running the test setup for a non-broadway backend.
When running the tests in CI, --suite=gtk overrides the default
exclude_suites, so we have to specify --no-suite=flaky and
--no-suite=failing explicitly.
This arrangement is inspired by GNOME/glib!2987, which was contributed
by Marco Trevisan.
Simon McVittie [Tue, 3 Jan 2023 10:22:42 +0000 (10:22 +0000)]
d/run-tests.sh: Use dh_auto_test
In particular this forces the C.UTF-8 locale, which should mean the tests
pass even when run under a non-UTF-8 locale like C (which official Debian
buildds do not do, but some other environments like reproducible-builds
might).
projects / gtk4.git / log